Packet prioritization and associated bandwidth and buffer management techniques for audio over ip

ABSTRACT

The present invention is directed to voice communication devices in which an audio stream is divided into a sequence of individual packets, each of which is routed via pathways that can vary depending on the availability of network resources. All embodiments of the invention rely on an acoustic prioritization agent that assigns a priority value to the packets. The priority value is based on factors such as whether the packet contains voice activity and the degree of acoustic similarity between this packet and adjacent packets in the sequence. A confidence level, associated with the priority value, may also be assigned. In one embodiment, network congestion is reduced by deliberately failing to transmit packets that are judged to be acoustically similar to adjacent packets; the expectation is that, under these circumstances, traditional packet loss concealment algorithms in the receiving device will construct an acceptably accurate replica of the missing packet. In another embodiment, the receiving device can reduce the number of packets stored in its jitter buffer, and therefore the latency of the speech signal, by selectively deleting one or more packets within sustained silences or non-varying speech events. In both embodiments, the ability of the system to drop appropriate packets may be enhanced by taking into account the confidence levels associated with the priority assessments.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.10/262,621 filed on Sep. 30, 2002 and entitled “PACKET PRIORITIZATIONAND ASSOCIATED BANDWIDTH AND BUFFER MANAGEMENT TECHNIQUES FOR AUDIO OVERIP”, the entire disclosure of which is herein incorporated by referencein its entirety.

FIELD OF THE INVENTION

The present invention relates generally to audio communications overdistributed processing networks and specifically to voice communicationsover data networks.

BACKGROUND OF THE INVENTION

Convergence of the telephone network and the Internet is driving themove to packet-based transmission for telecommunication networks. Aswill be appreciated, a “packet” is a group of consecutive bytes (e.g., adatagram in TCP/IP) sent from one computer to another over a network. InInternet Protocol or IP telephony or Voice Over IP (VoIP), a telephonecall is sent via a series of data packets on a fully digitalcommunication channel. This is effected by digitizing the voice stream,encoding the digitized stream with a codec, and dividing the digitizedstream into a series of packets (typically in 20 millisecondincrements). Each packet includes a header, trailer, and data payload ofone to several frames of encoded speech. Integration of voice and dataonto a single network offers significantly improved bandwidth efficiencyfor both private and public network operators.

In voice communications, high end-to-end voice quality in packettransmission depends principally on the speech codec used, theend-to-end delay across the network and variation in the delay (jitter),and packet loss across the channel. To prevent excessive voice qualitydegradation from transcoding, it is necessary to control whether andwhere transcodings occur and what combinations of codecs are used.End-to-end delays on the order of milliseconds can have a dramaticimpact on voice quality. When end-to-end delay exceeds about 150 to 200milliseconds one way, voice quality is noticeably impaired. Voicepackets can take an endless number of routes to a given destination andcan arrive at different times, with some arriving too late for use bythe receiver. Some packets can be discarded by computational componentssuch as routers in the network due to network congestion. When an audiopacket is lost, one or more frames are lost too, with a concomitant lossin voice quality.

Conventional VoIP architectures have developed techniques to resolvenetwork congestion and relieve the above issues. In one technique, voiceactivity detection (VAD) or silence suppression is employed to detectthe absence of audio (or detect the presence of audio) and conservebandwidth by preventing the transmission of “silent” packets over thenetwork. Most conversations include about 50% silence. When only silenceis detected for a specified amount of time, VAD informs the Packet VoiceProtocol and prevents the encoder output from being transported acrossthe network. VAD is, however, unreliable and the sensitivity of many VADalgorithms imperfect. To exacerbate these problems, VAD has only abinary output (namely silence or no silence) and in borderline casesmust decide whether to drop or send the packet. When the “silence”threshold is set too low, VAD is rendered meaningless and when too highaudio information can be erroneously classified as “silence” and lost tothe listener. The loss of audio information can cause the audio to bechoppy or clipped. In another technique, a receive buffer is maintainedat the receiving node to provide additional time for late andout-of-order packets to arrive. Typically, the buffer has a capacity ofaround 150 milliseconds. Most but not all packets will arrive before thetime slot for the packet to be played is reached. The receive buffer canbe filled to capacity at which point packets may be dropped. In extremecases, substantial, consecutive parts of the audio stream are lost dueto the limited capacity of the receive buffer leading to severereductions in voice quality. Although packet loss concealment algorithmsat the receiver can reconstruct missing packets, packet reconstructionis based on the contents of one or more temporally adjacent packetswhich can be acoustically dissimilar to the missing packet(s),particularly when several consecutive packets are lost, and thereforethe reconstructed packet(s) can have very little relation to thecontents of the missing packet(s).

SUMMARY OF THE INVENTION

These and other needs are addressed by the various embodiments andconfigurations of the present invention. The present invention isdirected generally to a computational architecture for efficientmanagement of transmission bandwidth and/or receive buffer latency.

In one embodiment of the present invention, a transmitter for a voicestream is provided that comprises:

(a) a packet protocol interface operable to convert one or more selectedsegments (e.g., frames) of the voice stream into a packet and

(b) an acoustic prioritization agent operable to control processing ofthe selected segment and/or packet based on one or more of (i) a levelof confidence that the contents of the selected segment are not theproduct of voice activity (e.g., are silence), (ii) a type of voiceactivity (e.g., plosive) associated with or contained in the contents ofthe selected segment, and (iii) a degree of acoustic similarity betweenthe selected segment and another segment of the voice stream.

The level of confidence permits the voice activity detector to provide aternary output as opposed to the conventional binary output. Theprioritization agent can use the level of confidence in the ternaryoutput, possibly coupled with one or measures of the traffic patterns onthe network, to determine dynamically whether or not to send the“silent” packet and, if so, use a lower transmission priority or classfor the packet.

The type of voice activity permits the prioritization agent to identifyextremely important parts of the voice stream and assign a highertransmission priorities and/or class to the packet(s) containing theseparts of the voice stream. The use of a higher transmission priorityand/or class can significantly reduce the likelihood that the packet(s)will arrive late, out of order, or not at all.

The comparison of temporally adjacent packets to yield a degree ofacoustic similarity permits the prioritization agent to controlbandwidth effectively. The agent can use the degree of similarity,possibly coupled with one or measures of the traffic patterns on thenetwork, to determine dynamically whether or not to send a “similar”packet and, if so, use a lower transmission priority or class for thepacket. Packet loss concealment algorithms at the receiver can be usedto reconstruct the omitted packet(s) to form a voiced signal thatclosely matches the original signal waveform. Compared to conventionaltransmission devices, fewer packets can be sent over the network torealize an acceptable signal waveform.

In another embodiment of the present invention, a receiver for a voicestream is provided that comprises:

(a) a receive buffer containing a plurality of packets associated withvoice communications; and

(b) a buffer manager operable to remove some of the packets from thereceive buffer while leaving other packets in the receive buffer basedon a level of importance associated with the packets.

In one configuration, the level of importance of the each of the packetsis indicated by a corresponding value marker. The level of importance orvalue marker can be based on any suitable criteria, including a level ofconfidence that contents of the packet contain voice activity, a degreeof similarity of temporally adjacent packets, the significance of theaudio in the packet to receiver understanding or fidelity, andcombinations thereof.

In another configuration, the buffer manager performs time compressionaround the removed packet(s) to prevent reconstruction of the packets bythe packet loss concealment algorithm. This can be performed by, forexample, resetting a packet counter indicating an ordering of thepackets, such as by assigning the packet counter of the removed packetto a packet remaining in the receive buffer.

In another configuration, the buffer manager only removes packet(s) fromthe buffer when the buffer delay or capacity equals or exceeds apredetermined level. When the buffer is not in an overcapacitysituation, it is undesirable to degrade the quality of voicecommunications, even if only slightly.

The various embodiments of the present invention can provide a number ofadvantages. First, the present invention can decrease substantiallynetwork congestion by dropping unnecessary packets, thereby providinglower end-to-end delays across the network, lower degrees of variationin the delay (jitter), and lower levels of packet loss across thechannel. Second, the various embodiments of the present invention canhandle effectively the bursty traffic and best-effort delivery problemscommonly encountered in conventional networks while maintainingconsistently and reliably high levels of voice quality reliably. Third,voice quality can be improved relative to conventional voice activitydetectors by not discarding “silent” packets in borderline cases.

These and other advantages will be apparent from the disclosure of theinvention(s) contained herein.

The above-described embodiments and configurations are neither completenor exhaustive. As will be appreciated, other embodiments of theinvention are possible utilizing, alone or in combination, one or moreof the features set forth above or described in detail below.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a simple network for a VoIP session betweentwo endpoints according to a first embodiment of the present invention;

FIG. 2 is a block diagram of the functional components of a transmittingvoice communication device according to the first embodiment;

FIG. 3 is a block diagram of the functional components of a receivingvoice communication device according to the first embodiment;

FIG. 4 is a flow chart of a voice activity detector according to asecond embodiment of the present invention;

FIG. 5 is a flow chart of a codec according to a third embodiment of thepresent invention;

FIG. 6 is a flow chart of a packet prioritizing algorithm according to asecond embodiment of the present invention;

FIG. 7 is a block diagram illustrating time compression according to afourth embodiment of the present invention; and

FIG. 8 is a flow chart of a buffer management algorithm according to thefourth embodiment of the present invention.

DETAILED DESCRIPTION

FIG. 1 is a simplistic VoIP network architecture according to a firstembodiment of the present invention. First and second voicecommunication devices 100 and 104 transmit and receive VoIP packets. Thepackets can be transmitted over one of two paths. The first and shortestpath is via networks 108 and 112 and router 116. The second and longerpath is via networks 108, 112, and 120 and routers 124 and 128.Depending upon the path followed, the packets can arrive at either ofthe communication devices at different times. As will be appreciated,network architectures suitable for the present invention can include anynumber of networks and routers and other intermediate nodes, such astranscoding gateways, servers, switches, base transceiver stations, basestation controllers, modems, router, and multiplexers and employ anysuitable packet-switching protocols, whether using connection orientedor connectionless services, including without limitation InternetProtocol or IP, Ethernet, and Asynchronous Transfer Mode or ATM.

As will be further appreciated, the first and second voice communicationdevices 100 and 104 can be any communication devices configured totransmit and/or receive packets over a data network, such as theInternet. For example, the voice communication devices 100 and 104 canbe a personal computer, a laptop computer, a wired analog or digitaltelephone, a wireless analog or digital telephone, intercom, and radioor video broadcast studio equipment.

FIG. 2 depicts an embodiment of a transmitting voice communicationdevice. The device 200 includes, from left to right, a first userinterface 204 for outputting signals inputted by the first user (notshown) and an outgoing voice stream 206 received from the first user, ananalog-to-digital converter 208, a Pulse Code Modulation or PMCinterface 212, an echo canceller 216, a Voice Activity Detector or VAD10, a voice codec 14, a packet protocol interface 18 and an acousticprioritizing agent 232.

The first user interface 204 is conventional and be configured in manydifferent forms depending upon the particular implementation. Forexample, the user interface 204 can be configured as an analog telephoneor as a PC.

The analog-to-digital converter 208 converts, by known techniques, theanalog outgoing voice stream 206 received from the first user interface204 into an outgoing digital voice stream 210.

The PCM interface 212, inter alia, forwards the outgoing digital voicestream 210 to appropriate downstream processing modules for processing.

The echo canceller 216 performs echo cancellation on the digital stream214, which is commonly a sampled, full-duplex voice port signal. Echocancellation is preferably G.165 compliant.

The VAD 10 monitors packet structures in the incoming digital voicestream 216 received from the echo canceller 216 for voice activity. Whenno voice activity is detected for a configurable period of time, the VAD10 informs the acoustic prioritizing agent 232 of the correspondingpacket structure(s) in which no voice activity was detected and providesa level of confidence that the corresponding packet structure(s)contains no meaningful voice activity. This output is typically providedon a packet structure-by-packet structure basis. These operations of theVAD are discussed below with reference to FIG. 4.

VAD 10 can also measure the idle noise characteristics of the first userinterface 204 and report this information to the packet protocolinterface 18 in order to relay this information to the other voicecommunication device for comfort noise generation (discussed below) whenno voice activity is detected.

The voice codec 14 encodes the voice data in the packet structures fortransmission over the data network and compares the acoustic information(each frame of which includes spectral information such as sound oraudio amplitude as a function of frequency) in temporally adjacentpacket structures and assigns to each packet an indicator of thedifference between the acoustic information in adjacent packetstructures. These operations are discussed below with reference to FIG.5. As shown in box 22, the voice codec typically include, in memory,numerous voice codecs capable of different compression ratios. Althoughonly codecs G.711, G,723.1, G.726, G.728, and G.78 are shown, it is tobe understood that any voice codec whether known currently or developedin the future could be in memory. Voice codecs encode and/or compressthe voice data in the packet structures. For example, a compression of8:1 is achievable with the G.78 voice codec (thus the normal 64 Kbps PCMsignal is transmitted in only 8 Kbps). The encoding functions of codecsare further described in Michaelis, Speech Digitization and Compression,in the International Encyclopedia of Ergonomics and Human Factors,edited by Warkowski, 2001; ITU-T Recommendation G.78 General Aspects ofDigital Transmission Systems, Coding of Speech at 8 kbit/s usingConjugate-Structure Algebraic-Code-Excited Linear-Prediction, March1996; and Mahfuz, Packet Loss Concealment for Voice Transmission Over IPNetworks, September 2001, each of which is incorporated herein by thisreference.

The prioritization agent 232 efficiently manages the transmissionbandwidth and the receive buffer latency. The prioritization agent (a)determines for each packet structure, based on the correspondingdifference in acoustic information between the selected packet structureand a temporally adjacent packet structure (received from the codec), arelative importance of the acoustic information contained in theselected packet structure to maintaining an acceptable level of voicequality and/or (b) determines for each packet structure containingacoustic information classified by the VAD 10 as being “silent” arelative importance based on the level of confidence (output by the VADfor that packet structure) that the acoustic information corresponds tono voice activity. The acoustic prioritization agent, based on thediffering levels of importance, causes the communication device toprocess differently the packets corresponding to the packet structures.The packet processing is discussed in detail below with reference toFIG. 6.

The packet protocol interface 18 assembles into packets and sequencesthe outgoing encoded voice stream and configures the packet headers forthe various protocols and/or layers required for transmission to thesecond voice communication device 300 (FIG. 3). Typically, voicepacketization protocols use a sequence number field in the transmitpacket stream to maintain temporal integrity of voice during playout.Under this approach, the transmitter inserts a packet counter, such asthe contents of a free-running, modulo-16 packet counter, into eachtransmitted packet, allowing the receiver to detect lost packets andproperly reproduce silence intervals during playout at the receivingcommunication device. In one configuration, the importance assigned bythe acoustic prioritizing agent can be used to configure the fields inthe header to provide higher or lower transmission priorities. Thisoption is discussed in detail below in connection with FIG. 6.

The packetization parameters, namely the packet size and the beginningand ending points of the packet are communicated by the packet protocolinterface 18 to the VAD 10 and codec 14 via the acoustic prioritizationagent 232. The packet structure represents the portion of the voicestream that will be included within a corresponding packet's payload. Inother words, a one-to-one correspondence exists between each packetstructure and each packet. As will be appreciated, it is important thatpacketization parameter synchronization be maintained between thesecomponents to maintain the integrity of the output of the acousticprioritization agent.

FIG. 3 depicts an embodiment of a receiving (or second) voicecommunication device 300. The device 300 includes, from right to left,the packet protocol interface 18 to remove the header information fromthe packet payload, the voice codec 14 for decoding and/or decompressingthe received packet payloads to form an incoming digital voice stream302, an adaptive playout unit 304 to process the received packetpayloads, the echo canceller 216 for performing echo cancellation on theincoming digital voice stream 306, the PCM interface 212 for performingcontinuous phase resampling of the incoming digital voice stream 316 toavoid sample slips and forwarding the echo cancelled incoming voicestream 316 to a digital-to-analog converter 308 that converts the echocancelled incoming voice stream 320 into an analog voice stream 324, andsecond user interface 312 for outputting to the second user the analogvoice stream 324.

The adaptive playout unit 304 includes a packet loss concealment agent328, a receive buffer 32, and a receive buffer manager 332. The adaptiveplayout unit 304 can further include a continuous-phase resampler (notshown) that removes timing frequency offset without causing packet slipsor loss of data for voice or voiceband modem signals and a timing jittermeasurement module (not shown) that allows adaptive control of FIFOdelay.

The packet loss concealment agent 328 reconstructs missing packets basedon the contents of temporally adjacent received packets. As will beappreciated, the packet loss concealment agent can perform packetreconstruction in a multiplicity of ways, such as replaying the lastpacket in place of the lost packet and generating synthetic speech usinga circular history buffer to cover the missing packet. Preferred packetloss concealment algorithms preserve the spectral characteristics of thespeaker's voice and maintain a smooth transition between the estimatedsignal and the surrounding original. In one configuration, packet lossconcealment is performed by the codec.

The receive buffer 32 alleviates the effects of late packet arrival bybuffering received voice packets. In most applications the receivebuffer 32 is a First-In-First-Out or FIFO buffer that stores voicecodewords before playout and removes timing jitter from the incomingpacket sequence. As will be appreciated, the buffer 32 can dynamicallyincrease and decrease in size as required to deal with late packets whenthe network is uncongested while avoiding unnecessary delays whennetwork traffic is congested.

The buffer manager 332 efficiently manages the increase in latency (orend-to-end delay) introduced by the receive buffer 32 by dropping (lowimportance) enqueued packets as set forth in detail below in connectionwith FIGS. 7 and 8.

In addition to packet payload decryption and/or decompression, the voicecodec 18 can also include a comfort noise generator (not shown) that,during periods of transmit silence when no packets are sent, generates alocal noise signal that is presented to the listener. The generatednoise attempts to match the true background noise. Without comfortnoise, the listener can conclude that the line has gone dead.

Analog-to-digital and digital-to-analog converters 208 and 308, thepulse code modulation interface 212, the echo canceller 216 a and b,packet loss concealment agent 328, and receive buffer 32 areconventional.

Although FIGS. 2 and 3 depict voice communication devices in simplexconfigurations, it is to be understood that each of the voicecommunication devices 200 and 300 can act both as a transmitter andreceiver in a duplexed configuration.

The operation of the VAD 10 will now be described with reference toFIGS. 2 and 4.

In the first step 60, the VAD 10 gets packet structure_(j) from the echocanceled digital voice stream 218. Packet structure counter i isinitially set to one. In step 64, the VAD 10 analyzes the acousticinformation in packet structure_(j) to identify by known techniqueswhether or not the acoustic information qualifies as “silence” or “nosilence” and determine a level of confidence that the acousticinformation does not contain meaningful or valuable acousticinformation. The level of confidence can be determined by knownstatistical techniques, such as energy level measurement, least meansquare adaptive filter (Widrow and Hoff 1959), and other StochasticGradient Algorithms. In one configuration, the acoustic threshold(s)used to categorize frames or packets as “silence” versus “nonsilence”vary dynamically, depending upon the traffic congestion of the network.The congestion of the network can be quantified by known techniques,such as by jitter determined by the timing measurement module (notshown) in the adaptive playout unit of the sending or receivingcommunication device, which would be forwarded to the VAD 10. Otherselected parameters include latency or end-to-end delay, number of lostor dropped packets, number of packets received out-of-order, processingdelay, propagation delay, and receive buffer delay/length. When theselected parameter(s) reach or fall below selected levels, the thresholdcan be reset to predetermined levels.

In step 68, the VAD 10 next determines whether or not packetstructure_(j) is categorized as “silent” or “nonsilent”. When packetstructure_(j) is categorized as being “silent”, the VAD 10, in step 412,notifies the acoustic prioritization agent 232 of the packetstructure_(j) beginning and/or endpoint(s), packet length, the “silent”categorization of packet structure, and the level of confidenceassociated with the “silent” categorization of packet structure. Whenpacket structure_(j) is categorized as “nonsilent” or after step 412,the VAD 10 in step 416 sets counter j equal to j+1 and in step 420determines whether there is a next packet structure_(j). If so, VAD 10returns to and repeats step 60. If not, VAD 10 terminates operationuntil a new series of packet structures is received.

The operation of the codec 14 will now be described with reference toFIGS. 2 and 5. In steps 500, 504 and 512, respectively, the codec 14gets packet structure, packet structure_(j−1), and packetstructure_(j+1). Packet structure counter j is, of course, initially setto one.

In steps 508 and 516, respectively, the codec 14 compares packetstructure_(j) with packet structure_(j−1), and packet structure_(j) withpacket structure_(j+1). As will be appreciated, the comparison can bedone by any suitable technique, either currently or in the future knownby those skilled in the art. For example, the amplitude and/or frequencywaveforms (spectral information) formed by the collective frames in eachpacket can be mathematically compared and the difference(s) quantifiedby one or more selected measures or simply by a binary output such as“similar” or “dissimilar”. Acoustic comparison techniques are discussedin Michaelis, et a., A Human Factors Engineer's Introduction to SpeechSynthesizers, in Directions in Human-Computer Interaction, edited byBadre, et al., 1982, which is incorporated herein by this reference. Ifa binary output is employed, the threshold selected for the distinctionbetween “similar” and “dissimilar” can vary dynamically based on one ormore selected measures or parameters of network congestion. Suitablemeasures or parameters include those set forth previously. When themeasures increase or decrease to selected levels the threshold is variedin a predetermined fashion.

In step 520, the codec 14 outputs the packet structuresimilarities/nonsimilarities determined in steps 508 and 516 to theacoustic prioritization agent 232. Although not required, the codec 14can further provide a level of confidence regarding the binary output.The level of confidence can be determined by any suitable statisticaltechniques, including those set forth previously. Next in step 524, thecodec encodes packet structure_(j). As will be appreciated, thecomparison steps 508 and 516 and encoding step 524 can be conducted inany order, including in parallel. The counter is incremented in step528, and in step 532, the codec determines whether or not there is anext packet structure_(j).

The operation of the acoustic prioritization agent 232 will now bediscussed with reference to with FIGS. 2 and 6.

In step 600, the acoustic prioritizing agent 232 gets packet_(j) (whichcorresponds to packet structure_(j)). In step 604, the agent 232determines whether VAD 10 categorized packet structure_(j) as “silence”.When the corresponding packet structure_(j) has been categorized as“silence”, the agent 232, in step 608, processes packet_(j) based on thelevel of confidence reported by the VAD 10 for packet structure_(j).

The processing of “silent” packets can take differing forms. In oneconfiguration, a packet having a corresponding level of confidence lessthan a selected silence threshold Y is dropped. In other words, theagent requests the packet protocol interface 18 to prevent packet_(j)from being transported across the network. A “silence” packet having acorresponding level of confidence more than the selected threshold issent. The priority of the packet can be set at a lower level than thepriorities of “nonsilence” packets. “Priority” can take many formsdepending on the particular protocols and network topology in use. Forexample, priority can refer to a service class or type (for protocolssuch as Differentiated Services and Internet Integrated Services), andpriority level (for protocols such as Ethernet). For example, “silent”packets can be sent via the assured forwarding class while “nonsilence”packets are sent via the expedited forwarding (code point) class. Thiscan be done, for example, by suitably marking, in the Type of Service orTraffic Class fields, as appropriate. In yet another configuration, avalue marker indicative of the importance of the packet to voice qualityis placed in the header and/or payload of the packet. The value markercan be used by intermediate nodes, such as routers, and/or by the buffermanager 332 (FIG. 3) to discard packets in appropriate applications. Forexample, when traffic congestion is found to exist using any of theparameters set forth above, value markers having values less than apredetermined level can be dropped during transit or after reception.This configuration is discussed in detail with reference to FIGS. 7 and8. Multiple “silence” packet thresholds can be employed for differingtypes of packet processing, depending on the application. As will beappreciated, the various thresholds can vary dynamically depending onthe degree of network congestion as set forth previously.

When the corresponding packet structure_(j) has been categorized as“nonsilence”, the agent 232, in step 618, determines whether the degreeof similarity between the corresponding packet structure_(j) and packetstructure_(j−1) (as determined by the codec 14) is greater than or equalto a selected similarity threshold X. If so, the agent 232 proceeds tostep 628 (discussed below). If not, the agent 232 proceeds to step 624.In step 624, the agent determines whether the degree of similaritybetween the corresponding packet structure_(j) and packetstructure_(j+1) (as determined by the codec 14) is greater than or equalto the selected similarity threshold X. If so, the agent 232 proceeds tostep 628.

In step 628, the agent 232 processes packet_(j) based on the magnitudeof the degree of similarity and/or on the treatment of the temporallyadjacent packet_(j−1). As in the case of “silent” packets, theprocessing of similar packets can take differing forms. In oneconfiguration, a packet having a degree of similarity more than theselected similarity threshold X is dropped. In other words, the agentrequests the packet protocol interface 18 to prevent packet_(j) frombeing transported across the network. The packet loss concealment agent328 (FIG. 3) in the second communication device 300 will reconstruct thedropped packet. In that event, the magnitude of X is determined by thepacket reconstruction efficiency and accuracy of the packet lossconcealment algorithm. If the preceding packet_(j−1) were dropped,packet_(j) may be forwarded, as the dropping of too many consecutivepackets can have a detrimental impact on the efficiency and accuracy ofthe packet loss concealment agent 328. In another configuration,multiple transmission priorities are used depending on the degree ofsimilarity. For example, a packet having a degree of similarity morethan the selected threshold is sent with a lower priority. The priorityof the packet is set at a lower level than the priorities of dissimilarpackets. As noted above, “priority” can take many forms depending on theparticular protocols and network topology in use. In yet anotherconfiguration, the value marker indicative of the importance of thepacket to voice quality is placed in the header and/or payload of thepacket. The value marker can be used as set forth previously and belowto cause the dropping of packets having value markers below one or moreselected marker value thresholds. Multiple priority levels can beemployed for multiple similarity thresholds, depending on theapplication. As will be appreciated, the various similarity and markervalue thresholds can vary dynamically depending on the degree of networkcongestion as set forth previously.

After steps 608 and 628 and in the event in step 624 that the similaritybetween the corresponding packet structure_(j) and packetstructure_(j+1) (as determined by the codec 14) is less than theselected similarity threshold X, the agent 232 proceeds to step 612. Instep 612, the counter j is incremented by one. In step 616, the agent232 determines whether there is a next packet_(j). When there is a nextpacket_(j), the agent 232 proceeds to and repeats step 600. When thereis no next packets, the agent 232 proceeds to step 632 and terminatesoperation until more packet structures are received for packetization.

The operation of the buffer manager 332 will now be described withreference to FIGS. 3 and 7-8. In step 800, the buffer manager 332determines whether the buffer delay (or length) is greater than or equalto a buffer threshold Y. If not, the buffer manager 332 repeats step800. If so, the buffer manager 332 in step 804 gets packet_(k) from thereceive buffer 32. Initially, of course the counter k is set to 1 todenote the packet in the first position in the receive buffer (or at thehead of the buffer). Alternatively, the manager 332 can retrieve thelast packet in the receive buffer (or at the tail of the buffer).

In step 808, the manager 332 determines if the packet is expendable;that is, whether the value of the value marker is less than (or greaterdepending on the configuration) a selected value threshold. When thevalue of the value marker is less than the selected value threshold, thepacket_(k) in step 812 is discarded or removed from the buffer and instep 816 the surrounding enqueued packets are time compressed around theslot previously occupied by packet_(k).

Time compression is demonstrated with reference to FIG. 7. The buffer 32is shown as having various packets 700 a-e, each packet payloadrepresenting a corresponding time interval of the voice stream. If themanager determines that packet 700 b (which corresponds to the timeinterval t₂ to t₃) is expendable, the manager 332 first removes thepacket 700 b from the queue 32 a and then moves packets 700 c-e ahead inthe queue. To perform time compression, the packet counters for packets700 c-e are decremented such that packet 700 c now occupies the timeslot t₂ to t₃, packet 700 d time slot t₂ to t₃, and packet 700 d timeslot t₃ to t₄. In this manner, the packet loss concealment agent 328will be unaware that packet 700 b has been discarded and will notattempt to reconstruct the packet. In contrast, if a packet is omittedfrom an ordering of packets, the packet loss concealment agent 328 willrecognize the omission by the break in the packet counter sequence. Theagent 328 will then attempt to reconstruct the packet.

Returning again to FIG. 8, the manager 332 in step 820 increments thecounter k and repeats step 800 for the next packet.

A number of variations and modifications of the invention can be used.It would be possible to provide for some features of the inventionwithout providing others.

For example in one alternative embodiment, the prioritizing agent'spriority assignment based on the type of “silence” detected can beperformed by the VAD 200.

In another alternative embodiment though FIG. 2 is suitable for use witha VoIP architecture using Embedded Communication Objects interworkingwith a telephone system and packet network, it is to be understood thatthe configuration of the VAD 10, codec 14, prioritizing agent 232 and/orbuffer manager 332 of the present invention can vary significantlydepending upon the application and the protocols employed. For example,the prioritizing agent 232 can be included in an alternate location inthe embodiment of FIG. 2, and the buffer manager in an alternatelocation in the embodiment of FIG. 3. The prioritizing agent and/orbuffer manager can interface with different components than those shownin FIG. 2 for other types of user interfaces, such as a PC, wirelesstelephone, and laptop. The prioritizing agent and/or buffer manager canbe included in an intermediate node between communication devices, suchas in a switch, transcoding device, translating device, router, gateway,etc.

In another embodiment, the packet comparison operation of the codec isperformed by another component. For example, the VAD and/or acousticprioritization agent performs these functions.

In another embodiment, the level of confidence determination of the VADis performed by another component. For example, the codec and/oracoustic prioritization agent performs these functions.

In yet a further embodiment, the codec and/or VAD, during packetstructure processing attempt to identify acoustic events of greatimportance, such as plosives. When such acoustic events are identified(e.g., when the difference identified by the codec exceeds apredetermined threshold), the acoustic prioritizing agent 232 can causethe packets corresponding to the packet structures to have extremelyhigh priorities and/or be marked with value markers indicating that thepacket is not to be dropped under any circumstances. The loss of apacket containing such important acoustic events often cannot bereconstructed accurately by the packet loss concealment agent 328.

In yet a further embodiment, the analyses performed by the codec, VAD,and acoustic prioritizing agent are performed on a frame level ratherthan a packet level. “Silent” frames and/or acoustically similar framesare omitted from the packet payloads. The procedural mechanisms forthese embodiments are similar to that for packets in FIGS. 4 and 5. Infact, the replacement of “frame” for “packet structure” and “packet” inFIGS. 4 and 5 provides a configuration of this embodiment.

In yet another embodiment, the algorithms of FIGS. 6 and 8 are statedriven. In other words, the algorithms are not triggered until networkcongestion exceeds a predetermined amount. The trigger for the state tobe entered can be based on any of the performance parameters set forthabove increasing above or decreasing below predetermined thresholds.

In yet a further embodiment, the dropping of packets based on the valueof the value marker is performed by an intermediate node, such as arouter. This embodiment is particularly useful in a network employingany of the Multi Protocol Labeling Switching, ATM, and IntegratedServices Controlled Load and Differentiate Services.

In yet a further embodiment, the positions of the codec and adaptiveplayout unit in FIG. 3 are reversed. Thus, the receive buffer 32contains encoded packets rather than decoded packets.

In yet a further embodiment, the acoustic prioritization agent 232processes packet structures before and/or after encryption.

In yet a further embodiment, a value marker is not employed and thebuffer manager itself performs the packet/frame comparison to identifyacoustically similar packets that can be expended in the event thatbuffer length/delay reaches undesired levels.

In other embodiments, the VAD 10, codec 14, acoustic prioritizationagent 232, and/or buffer manager 332 are implemented as software and/orhardware, such as a logic circuit, e.g, an Application SpecificIntegrated Circuit or ASIC.

The present invention, in various embodiments, includes components,methods, processes, systems and/or apparatus substantially as depictedand described herein, including various embodiments, subcombinations,and subsets thereof. Those of skill in the art will understand how tomake and use the present invention after understanding the presentdisclosure. The present invention, in various embodiments, includesproviding devices and processes in the absence of items not depictedand/or described herein or in various embodiments hereof, including inthe absence of such items as may have been used in previous devices orprocesses, e.g., for improving performance, achieving ease and\orreducing cost of implementation.

The foregoing discussion of the invention has been presented forpurposes of illustration and description. The foregoing is not intendedto limit the invention to the form or forms disclosed herein. Althoughthe description of the invention has included description of one or moreembodiments and certain variations and modifications, other variationsand modifications are within the scope of the invention, e.g., as may bewithin the skill and knowledge of those in the art, after understandingthe present disclosure. It is intended to obtain rights which includealternative embodiments to the extent permitted, including alternate,interchangeable and/or equivalent structures, functions, ranges or stepsto those claimed, whether or not such alternate, interchangeable and/orequivalent structures, functions, ranges or steps are disclosed herein,and without intending to publicly dedicate any patentable subjectmatter.

1. A system for transmitting voice communications over a data network,comprising: (a) an input operable to receive a voice stream from a user,the voice stream comprising a plurality of temporally distinct segments;(b) a packet protocol interface operable to convert at least oneselected first segment of the voice stream into at least a first packet;and (c) an acoustic prioritization agent operable to control processingof at least one of the first segment and the at least a first packetbased on at least one of a level of confidence that the contents of theselected first segment are not the product of voice activity, a type ofvoice activity associated with the contents of the first segment, and adegree of acoustic similarity between the first segment and a secondsegment of the voice stream.
 2. The system of claim 1, wherein theacoustic prioritization agent is operable to assign an importance to theat least one of the first segment and the at least a first packet basedon the at least one of the level of confidence, type of voice activityand the degree of acoustic similarity.
 3. The system of claim 2, whereinthe importance is a value marker and the packet protocol interface isoperable to incorporate the value marker into the at least a firstpacket.
 4. The system of claim 2, wherein the importance is a serviceclass assigned to the at least a first packet.
 5. The system of claim 2,wherein the importance is a transmission priority assigned to the atleast a first packet.
 6. The system of claim 1, wherein the packetprotocol interface is operable to not transmit the at least a firstpacket when the at least one of the level of confidence and the degreeof acoustic similarity is one of less than and greater than apredetermined threshold.
 7. The system of claim 6, wherein thepredetermined threshold is varied based on at least one of jitter,latency, a number of missing packets, a number of packets receivedout-of-order, a processing delay, a propagation delay, and a receivebuffer length.
 8. The system of claim 1, wherein the at least one of thelevel of confidence, type of voice activity and the degree of acousticsimilarity is the level of confidence.
 9. The system of claim 1, whereinthe at least one of the level of confidence, type of voice activity andthe degree of acoustic similarity is the degree of acoustic similarity.10. The system of claim 9, wherein the second segment temporallyprecedes the first segment and a third segment of the voice streamtemporally follows the first segment and further comprising: a codecoperable to compare the first segment with the second segment of thevoice stream to determine a first degree of acoustic similarity betweenthe first and second segments and compare the first segment with thethird segment of the voice stream to determine a second degree ofacoustic similarity between the first and third segments.
 11. The systemof claim 10, wherein the prioritization agent controls processing of theat least one of the first segment and the at least a first packet basedon at least one of the first and second degrees of acoustic similarityone of exceeding or being less than a selected similarity threshold. 12.The system of claim 1, wherein the first segment corresponds to apayload of the at least a first packet.
 13. The system of claim 1,wherein the first segment corresponds to a frame of the at least a firstpacket.
 14. The system of claim 1, wherein different classes of servicesare used for different segments of the voice stream.
 15. The system ofclaim 1, wherein the packet protocol interface is operable to usedifferent transmission priorities for different segments of the voicestream.
 16. The system of claim 1, wherein the at least one of the levelof confidence, type of voice activity and the degree of acousticsimilarity is the type of voice activity.
 17. The system of claim 16,wherein the type of voice activity is a plosive.
 18. The system of claim9, wherein the at least a packet is not transmitted and furthercomprising: a packet loss concealment agent operable to laterreconstructing the first segment.
 19. The system of claim 3, furthercomprising: a buffer manager operable to remove the at least a firstpacket from a receive buffer when the value of the value marker is oneof less than and greater than a predetermined value threshold.
 20. Amethod for transmitting voice communications over a data network,comprising: receiving a voice stream from a user, the voice streamcomprising a plurality of temporally distinct segments; converting atleast one selected first segment of the voice stream into at least afirst packet; and processing at least one of the first segment and theat least a first packet based on at least one of a level of confidencethat the contents of the selected first segment are not the product ofvoice activity, a type of voice activity associated with the contents ofthe first segment, and a degree of acoustic similarity between the firstsegment and a second segment of the voice stream.